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7.2.7 Almost Sure Convergence 

Consider a sequence of random variables Xi, X2, X3, • ■ ■ that is defined on an underlying sample space S . For simplicity, let us assume 
that S' is a finite set, so we can write 

S {$1 5 S2 j * * ’ 5 Sfc }• 

Remember that each X n is a function from S to the set of real numbers. Thus, we may write 

X n ( 5 j) x nii f°r i 1 , 2 , • • •, k. 

After this random experiment is performed, one of the Si 's will be the outcome of the experiment, and the values of the X n 's are known. If 
Sj is the outcome of the experiment, we observe the following sequence: 

X \ j , X 2j - X 3j: • ' ■ • 

Since this is a sequence of real numbers, we can talk about its convergence. Does it converge? If yes, what does it converge to? Almost sure 
convergence is defined based on the convergence of such sequences. Before introducing almost sure convergence let us look at an example. 

Example 7.12 

Consider the following random experiment: A fair coin is tossed once. Here, the sample space has only two elements S = {H,T}. We 
define a sequence of random variables X 1 , X 2 > X 3 , • • • on this sample space as follows: 

in TT iis = H 

X n (s) = l 

[(-1)" if s = T 

a. For each of the possible outcomes (H or T), determine whether the resulting sequence of real numbers converges or not. 

b. Find 


P (jsi G S : Jim X n (si) = 1 j) . 

• Solution 

o a. If the outcome is H , then we have X n ( H ) = so we obtain the following sequence 

12 3 4 
2’ 3’ 4’ 5’"'' 

This sequence converges to 1 as n goes to infinity. If the outcome is T, then we have X n (T) = ( — 1)”, so we obtain 
the following sequence 


- 1 , 1 ,- 1 , 1 ,- 1 ,..-. 

This sequence does not converge as it oscillates between — 1 and 1 forever, 
b. By part (a), the event {s^ G S : X n (s^) = 1} happens if and only if the outcome is H, so 

P ({s 8 G 5 : Jim X n (8i) = l}) = P(H) 

_ 1 

“ 2 ' 


In the above example, we saw that the sequence X n (s) converged when S — H and did not converge when S — T. In general, if the 
probability that the sequence X n (s) converges to A(s) is equal to 1, we say that X n converges to X almost surely and write 

a.s. 

x n y X. 


https://www.probabilitycourse.com/chapter7/7_2_7_almost_sure_convergence.php 


1/5 



9/17/2018 


Almost Sure Convergence 


Almost Sure Convergence 

A sequence of random variables X\, X 2 , X 3 , ■ ■ ■ converges almost surely to a random variable X , shown by X n 

p({seS: lim X n (s) = X(s)}) = 1. 


a.s. 


Example 7.13 

Consider the sample space S = [0, 1] with a probability measure that is uniform on this space, i.e., 

P([a, b]) — b — a, for all 0 < a < b < 1. 

Define the sequence •[ X n , Tl — 1, 2, • • • } as follows: 

X n {s) = l 

f 0 otherwise 

Also, define the random variable X on this sample space as follows: 

1 

X(s) 


n+1 

2 n 


0 <s<l 


0 otherwise 


Show that X n 


X. 


Solution 

o Define the set A as follows: 

A — {s € S : lim X n (s ) = X(s)j. 

L n—>oo J 

We need to prove that P( A) — 1. Let's first find A. Note that > ^-.soforanyS G [0, y), we have 

*»(«) =* X{s) = 1. 

Therefore, we conclude that [0, 0.5) C A.NowifS > -i-.then 

X(s) =S 0. 

Also, since 2s — 1 > 0 , we can write 

1 


X n (s ) = 0, for all n > 


2 s - 1 


Therefore, 


lim X n (s ) = 0 = X(s), for all s > —. 

n—Kx> 2 


We conclude (j, 1] C S. You can check that S = j A, since 


1 - 1 = 1 , 


for all: 


while X (h) = 0. We conclude 
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Since P{A) 



a.s. 

1, we conclude X n -> 


In some problems, proving almost sure convergence directly can be difficult. Thus, it is desirable to know some sufficient conditions for 
almost sure convergence. Here is a result that is sometimes useful when we would like to prove almost sure convergence. 


Theorem 7.5 

Consider the sequence X\, Xi, X$, • • •. If for all e > o, we have 

oo 

Y,P(\X n -X\ > e) < oo, 

n= 1 


a.s. 

then X n - X. 

Example 7.14 

Consider a sequence { X n , n 


1, 2, 3, • • •} such that 


X n = 



with probability j 
with probability j 


a.s. 

Show that X n - > 0. 

• Solution 

o By the Theorem above, it suffices to show that 


Note that \X n | 


oo 

]TP(|X»| > e) < OO. 

n=1 

. Thus, | X n | > e if and only if n < ^ . Thus, we conclude 


oo LtJ 

J2 P (\Xn\ >e) <^2P{\X n \ >e) 

n= 1 n=1 


= L-J< 

e 


Theorem 7.5 provides only a sufficient condition for almost sure convergence. In particular, if we obtain 

oo 

J^P(\X n -X\ >e) =00, 

n= 1 

then we still don't know whether the X n 's converge to X almost surely or not. Here, we provide a condition that is both necessary and 
sufficient. 

Theorem 7.6 

Consider the sequence X\, X 2 , X%, ■ • •. For any e > 0, define the set of events 

A m — {\X n — X\ < e, for all n > m}. 
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X if and only if for any e > 0, we have 

lim P(A m ) — 1. 

ra—xx> 


Example 7.15 

Let X i, X- 2 , X%, • • • be independent random variables, where X n ~ Bernoulli (-^-) for n — 2, 3, • • •. The goal here is to check 
a.s. 

whether X n -> 0. 

1. Check that P(|X n | > e) * OO. 

2. Show that the sequence X±, X 2 , ■ ■ . does not converge to 0 almost surely using Theorem 7.6 
• Solution 

o 1. We first note that for 0 < e < 1, we have 

OO 

E p (i x ”i >e ) 

n=1 


2. To use Theorem 7.6, we define 

A m = {\X n \ < e, for all n > m}. 

Note that for 0 < € < 1, we have 

A m — {X n = 0, for all n > m}. 

According to Theorem 7.6 , it suffices to show that 

lim P(A m ) < 1. 

ra—>oo 

We can in fact show that limjn-^oo P(A m ) — 0. To show this, we will prove P(A m ) — 0, for every m > 2. For 
0 < e < 1 , we have 

P(A m ) ~ P({X n ~ 0, for all n > m}) 

< P(y{X n — 0, for n — m, m + 1, • • •, TV}) (for every positive integer N > m) 
— P{X m — 0)P(X m+ i = 0) • • • P(Xn — 0) (since the Xj s are independent) 
m — 1 m N — 1 

m m +1 N 

m — 1 

“ N 

Thus, by choosing N large enough, we can show that P(j4 m ) is less than any positive number. Therefore, 

P(A m ) — 0 for all m > 2. We conclude that lim,,^;^ P(^4 m ) = 0. Thus, according to Theorem 7.6, the 
sequence X\, X 2 , ■ ■ ■ does not converge to 0 almost surely. 




An important example for almost sure convergence is the strong law of large numbers (SLLN). Here, we state the SLLN without proof. The 
interested reader can find a proof of SLLN in [19], A simpler proof can be obtained if we assume the finiteness of the fourth moment. (See 
[20] for example.) 

The strong law of large numbers (SLLN) 

Let Jf 'd-d. random variables with a finite expected value = /i < OO. Let also 
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M n 


X^Xz + .-.+Xn 
n 


Then M n 


M- 


We end this section by stating a version of the continuous mapping theorem. This theorem is sometimes useful when proving the 
convergence of random variables. 


Theorem 7.7 Let X\, X 2 , X 3 , • • • be a sequence of random variables. Let also h : R 1 —> ]R be a continuous function. Then, the 
following statements are true: 

d d 

1. IfX„ — / X, then h(X n ) — h(X). 

2. If X n — X,thenh(X n ) —■/ /l(X). 

3. If X n ^ X, then h{X n ) ^ h{X). 

*— previous 
next —» 


https://www.probabilitycourse.com/chapter7/7_2_7_almost_sure_convergence.php 


5/5 



